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REAL PARTY IN INTEREST 

The real party in interest in this appeal is the following party: International Business Machines 
Corporation of Aimonk, New York. 
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RELATED APPEALS AND INTERFERENCES 



With respect to other appeals or interferences that will directly affect, or be directly affected by 3 or 
have a bearing on the Board's decision in the pending appeal, there are no such appeals or 
interferences. 
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STATUS OF CLAIMS 



A. TOTAL NUMBER OF CLAIMS IN APPLICATION 

Claims in the application axe: 1-33 



B. STATUS OF ALL THE CLAIMS IN APPLICATION 

1. Claims canceled; NONE 

2. Claims withdrawn from consideration but not canceled: NONE 
3» Claims pending: 1-33 

4. Claims allowed: NONE 

5 . Claims rej ected: 1-33 

6. Claims objected to: NONE 

C. CLAIMS ON APPEAL 

The claims on appeal are: 1 -33 
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RECEIVED 

CENTRAL PAX CENTER 

FEB 2 I 2006 

STATUS OF AMENDMENTS 

An Amendment after Final Rejection was not filed Therefore, Claims 1-33 on appeal herein are 
as amended in the Response to Office Action filed July 1, 2005, 
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SUMMARY OF CLAIMED SUBJECT MATTER 
A. CLAIM 1 - INDEPENDENT 

Claim 1 is directed to a method in a data processing system for presenting graphical data to a 
user. A set of graphical data is analyzed to determine a set of critical factors present in the 
graphical data to form determined critical factors (Specification, pg. 10, line 6 - pg. 1 1, lane 14; 
Figure 3, step 310, Figure 7). The determined critical factors are ranked according to respective 
priorities set for each of the critical factors (Specification, pg. 9 f line 16 - pg. 10, line 9; Figure 
3, step 308, Figures 5A and 5B), Then a textual description of the set of graphical data, ordered 
according to the priorities of each of the respective critical factors, is generated (Specification, 
pg. 12, line 7 - pg. 13, line 8; Figure 3, step 312), 

A. CLAIM 8 - INDEPENDENT 

Claim 8 is directed to a computer program product for presenting graphical data to a user. The 
computer program product includes a set of instructions for performing the step of analyzing a 
set of graphical data to determine a set of critical factors present in the graphical data to form 
determined critical factors (Specification, pg. 10, line 6 - pg. 1 1, line 14; Figure 3, step 310, 
Figure 7). The computer program product also includes a set of instructions for performing the 
step of ranking the determined critical factors according to respective priorities set for each of 
the critical factors (Specification, pg. 9, line 16 - pg. 10, line 9; Figure 3, step 308, Figures 5A 
and SB). The computer program product also includes a set of instructions for generating a 
textual description of the set of graphical data, ordered according to the priorities of each of the 
respective critical factors (Specification, pg. 12, line 7 - pg. 13, line 8; Figure 3, step 312). 

A. CLAIM IS - INDEPENDENT 

Claim IS is directed to a data processing system. The data processing system has circuitry 
(Figure 2, 210, 212, 214, and 216) operable for analyzing a set of graphical data to determine a 
set of critical factors present in the graphical data to form determined critical factors 
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(Specification, pg. 10, line 6 - pg. 1 1, line 14; Figure 3, step 310, Figure 7). The data 
processing system has circuitry (Figure 2, 210, 212, 214, and 216) operable for ranking the 
determined critical factors according to respective priorities sot for each of the critical factors 
(Specification, pg. 9, line 16 - pg. 10, line 9; Figure 3, step 308, Figures SA and SB). The data 
processing system has circuitry (Figure 2, 210, 212, 214, and 216) operable for generating a 
textual description of the set of graphical data, ordered according to the priorities of each of the 
respective critical factors (Specification, pg. 12, line 7 - pg. 13, line 8; Figure 3, step 312). 

Al. CLAIM 2 -DEPENDENT 

Claim 2 is directed to a method in a data processing system for presenting graphical data to a 
user, the same method as recited in claim 1, wherein the set of critical factors and the textual 
description are selected according to a selected mode (Specification, pg^ 9, line 16 - pg. 10, line 
9; Figure 3, step 308, Figures 5A and SB), 

Al. CLAIM 9 - DEPENDENT 

Claim 9 is directed to a computer program product for presenting graphical data to a user, the 
same computer program product as recited in claim 8, wherein the set of critical factors and the 
textual description are selected according to a selected mode (Specification, pg. 9„ line 16 - pg. 
10, line 9; Figure 3, step 308, Figures 5A and 5B). 

Al. CLAIM 16 - DEPENDENT 

Claim 16 is directed to a data processing system for presenting graphical data to a user, the same 
method as recited in claim 15, wherein the set of critical factors and the textual description are 
selected according to a selected mode (Specification, pg. 9, line 16 - pg. 10, line 9; Figure 3, 
step 308, Figures 5A and SB). 

A2. CLAIM 6 - DEPENDENT 

Claim 6 is directed to a method in a data processing system for presenting graphical data to a 
user, the same method as recited in claim 2, wherein the priorities of each of the respective 
critical factors is determined in accordance with the selected mode (Specification, pg. 9, line 16 - 
pg. 10, line 9; Figure 3, step 308, Figures 5A and SB). 
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A2. CLAIM 13 - DEPENDENT 

Claim 13 is directed to a computer program product for presenting graphical data to a user, the 
same computer program product as recited in claim 9, wherein the priorities of each of the 
respective critical factors is determined in accordance with the selected mode (Specification, pg. 
9, liae 16 - pg, 10, line 9; Figure 3, step 308, Figures 5 A and 5B). 

A2. CLAIM 20 - DEPENDENT 

Claim 20 is directed to a data processing system for presenting graphical data to a user, the same 
method as recited in claim 16, wherein the priorities of each of the respective critical factors is 
determined in accordance with the selected mode (Specification, pg. 9, line 16 - pg. 10, line 9; 
Figure 3, step 308, Figures 5A and SB). 
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GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

A. GROUND OF REJECTION 1 (Claims 1, 2, 6, 22, 23, 31, 8, 9, 13, 24, 32, 15, 16, 20, 
25, 26 and 33) 

Claims 1, 2, 6, 22, 23, 31> 8, 9, 13, 24, 32, 15, 16, 20, 25, 26 and 33 stand rejected under 35 U.S.C. 
§ 103(a) as being unpatentable over LL S. Patent 5,75 1,286, to Barber et al. in view of U. S- Patent 
No. 6,501,779, to McLaughlin et al. 

B. GROUND OF REJECTION 2 (Claims 3, 10 and 17) 

Claims 3, 10 and 17 stand rejected under 35 US.C. § 103(a) as being unpatentable over U. S. 
Patent 5,751,286, to Barber et aL in view of U. S. Patent No, 6,501,779, to McLaughlin et al. and 
further in view of applicant's admitted prior art. 

C. GROUND OF REJECTION 3 (Claims 4, 5, 11, 12, 18 and 19) 

Claims 4, 5, 11, 12, 18 and 19 stand rejected under 35 U.S.C, § 103(a) as being unpatentable over 
XL S. Patent 5,751,286, to Barber et al. in view of U. S. Patent No. 6,501,779, to McLaughlin et al. 
and further in view of U. S. Patent No. 5,736,978, to Hasser et al. 

D. GROUND OF REJECTION 4 (Claims 7, 14 and 21) 

Claims 7 3 14, and 21 stand rejected under 35 U.S.C § 103(a) as being unpatentable overU. S. 
Patent 5,751,286, to Barber et aL in view of U. S. Patent No. 6,501,779, to McLaughlin et al. and 
further in view of U. S. Patent No, 6,370,566, to Discolo et al. 

E. GROUND OF REJECTION 5 (Claims 27-30) 

Claims 27-30 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over U. S. Patent 
5,751,286, to Barber et al. in view of U S. Patent No. 6,501,779, to McLaughlin et al. and further 
in view of applicant's admitted prior art. 
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ARGUMENT 



A, GROUND OF REJECTION 1 (Claims 1, 2, 6, 22, 23, 31, 8, 9, 13, 24, 32, 15, 16, 20, 
25, 26 and 33) 

In rejecting the claims as being unpatentable over Barber in view of McLaughlin, the 
Office Action states the following: 

As per claims 1 > 8 and 15, Barber teaches a computer implemented method 
and corresponding syBtem for presenting graphical data to a user comprising the 
steps/means: 

analyzing a set of graphical data to determine a set of critical factors present 
in the graphical data to form determined critical factors (coL 6, line 30 - col. 7, line 
13); 

ranking the determinjed critical factors according to respective priorities set 
for each of the critical factors (col. 14, lines 44-67); and 

Barber teaches generating a set of graphical data, ordered according to the 
priorities of each of the respective critical factor (coL 14, lines 65-67 and col. 9 3 
lines 57-61), However, Barber does not disclose a textual description of the set of 
graphical data. McLaughlin teaches the web/test translator converts text, graphic 
icons, menu options, and communication links within a web site into equivalent 
forms of text data (e.g, coL 4, lines 3-9). It would have been obvious to an artisan at 
the time of the invention to use the teaching from McLaughlin of providing a textual 
description of the set of graphical data in Barber's system since it would enable 
Barber's system to be used b sight impaired people. 

Final Office Action dated September 22, 2005, pages 2 and 3. 

Furthermore the Office Action also states: 

(a) The thumbnails are not equated as being equivalent to "critical factors". 
In fact, the critical factors such as color, texture, size, shape and layout (e.g., coL 5, 
lines 57-59) in each of the image in the set of graphical data from the database 
(database 36 of fig, 1) are ranked according to the respective priority set for each of 
the critical factors. 

(b) The generated results list 80 is, in fact, a set of images (graphical data) 
ordered according to the priorities of each of the respective critical factors set in the 
image query construction (e.g., col. 8, lines 46-55). 

(c) As explained above in (b) Barber does teaches generating the set of 
graphical data ordered according to the priorities of each of the respective critical 
factors. However, Barber does not disclose a textual description of the set of 
graphical data, McLaughlin teaches the web/text translator converts text, graphic 
icons* menu options, and communication links within a web site into equivalent 

(Appeal Brief Page 10 of 33) 
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forms of text data (e.g. col 4, lines 3-9), It would have been obvious to an artisan at 
the time of the invention to use the teaching from McLaughlin of providing a textual 
description of the set of graphical data in Barber's system since it would enable 
Barber's system to be used by sight impaired people. 

Final Office Action dated September 22, 2005, page 8. 

Claim 1 of the present application reads as follows: 

1 . A method in a data processing system for presenting graphical data to a 
user, comprising the steps of: 

analyzing a set of graphical data to determine a set of critical factors 
present in the graphical data to form determined critical factors; 

ranking the determined critical factors according to respective priorities set 
for each of the critical factors; and 

generating a textual description of the set of graphical data, ordered 
according to the priorities of each of the respective critical factors. 

Barber does not teach or suggest all of the features in claim 1. Barber teaches a process 
wherein an image query is built from thumbnails that are dropped into a selection window. The 
thumbnails are definitions of image characteristics. The query engine converts the information 
from the thumbnails into image characteristic values. Thus, the query constructed is in terms of 
the values of the image characteristics of interest, (see Barber, col. 6, line 30 - col. 7, line 1 3). 
Each image to be queried has a stored data representation in which the image is broken into areas 
of interest, or masks. The calculated values for the image characteristics of each mask are stored 
in the data representation, (see Barber, col. 6, line 30 - col. 7, line 13). When a query is 
performed, the image characteristic values obtained from the thumbnails are compared to the 
values for the corresponding image characteristics contained in each mask of the data 
representation. This gives a composite distance score for each mask, or area of the image. The 
scores for all areas are tallied, yielding a composite distance, or global score. The stored images 
are then ranked by this total score, (see Barber, col 7, line 47 - col. 8, line 48; and col. 14, lines 
47 -66). 

Claim 1 of the present invention recites the feature of ranking the determined critical 
factors according to respective priorities set for each of the critical factors . The Examiner alleges 
that this feature is found in column 14, lines 44 through 67 and column 7, lines 6 through 25, of 
Barber, which are reproduced below: 

(Appeal Brief Page 1 1 of 33) 
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(i) For each image in the collection, compute its similarity score: 

(a) For each area specified in the query, compute a positional feature score 
that compares the area's similarity to the image areas computed in Step 
3. This score combine both features along with positional similarity so 
that areas with similar features get higher scores, dissimilar features 
get lower scores, and areas positionaUy close get higher scores, and 
areas positionaUy far get lower scores. The result is a score, for each 
query area, of its positional feature similarity within tins image. The 
highest scores will be obtained by areas both positionaUy close and 
with similar features. Indexing techniques could be used to increase 
the performance of searching for the "best" matches. 

(b) Combine the scores for all query areas to give a global score for the 
image. 

(c) Rank the images by their global scores and return, as the results of the 
query, the images with the best scores, 

(Barber, col. 14, lines 44-67) 

Later, when a query is assembled, an object/thumbnail procedure described below 
is employed to construct a description (a "sample image") of the images which a 
user wishes to retrieve from the image database, with the query being constructed 
in terms of values of the image characteristics of interest. The query is used to 
find images in the database with image characteristic values that are similar to 
those included in the sample image, To do a query, the QBIC engine 32 converts 
pictorial query information (e.g,, the information from the thumbnails and their 
location) from the image query window into image characteristic values. 

FIG, 4 illustrates diagrammatically the essential process for image query 
construction and execution where the query is composed by dragging and 
dropping thumbnails. InitiaUy, a query is assembled by dragging and dropping 
characteristic thumbnails in the image query window 23. The deposit of a 
thumbnail creates an object 70. A thumbnail may overlap other thumbnails within 
the same object For example, in the object 70, thumbnails for shape (in 
SPAKMS, such as Sl.sub.q, S2,$ub.q, S3.sub.q, . . . ), category (spires), color (in 
color components, such as R.sub,q, G.sub.q, B.sub.q), and texture (in TP ARMS, 
such as Tl.sub.q, T2_sub.q, T3.sub.q) overlay each other within a bounding 
rectangle which defines the object 70. 

(Barber, col. 7, lines 2-26) 

The first cited passage, column 14, lines 44 through 67, teaches getting a global score for 
an image and then ranking the images examined by this total score. While the above cited 
passage does teach ordering graphical data, the passage does not teach ranking the determined 
critical factors according to respective priorities set for each of the critical factors. The sorting of 
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images from best match to worst match is achieved by ranking the images' global scores. A 
global score is neither a critical factor nor is it a respective priority set for each of the critical 
factors. Rather, a global score is the sum of the relational scores for each item of data in a set of 
graphical data compared to a corresponding item of data in a base set of graphical data, (see 
Barber, col 14, lines 44-64). The results arrived at under Barber are ordered according to a total 
score, which is based on several factors, but the factors themselves are not prioritized. That is, as 
taught by Barber, no single factor is more important to have than any other, as the total score for 
each image is used to rank the images, rather than ranking the factors. 
Barber, in column 6, line 30 through column 7, line 13, states; 



Thumbnail data representations (definitions) which are used in the first 
embodiment of the invention are stored as described with reference to FIG. 1 9 and 
have the general foim illustrated in FIG. 3. In FIG. 3, thumbnail definitions for 
color, texture, size, shape, and category are indicated, respectively, by reference 
number 60, 62, 64* 66 and 68. Each thumbnail definition represents one of a 
plurality of thumbnail definitions for a particular referenced characteristic. For 
example, the thumbnail definition 60 is a data representation for a color thumbnail 
which maybe denoted as a thumbnail M representing a color M. In this regard, M 
would be the identity of a thumbnail contained in a color characteristic window 
presented on the display 13 in FIG* 1 during query construction. The definition 
for the color thumbnail M is a data structure indexed by the identity of the color 
thumbnail and including average intensity values for the red, green and blue 
component of the color M. The definition 62 for texture thumbnail P denoting a 
smooth texture includes the identity (P) for the smooth texture thumbnail and a 
data representation for a smooth texture. The data representation can, for example, 
be represented by a set of texture parameters including coarseness, contrast and 
directionality. Similarly, the size thumbnail Q represents the size of a pixellated 
area; the shape thumbnail S can include a data representation using an algebraic 
moment invariant; and. the spire category thumbnail includes, for example, the 
text "spire" or additional textural annotation. Note that a layout thumbnail is not 
required since the location of the bounding rectangle of amy thumbnail in the 
image query window 23 would be available through the window control 22 to the 
query engine 32, 

It should be evident that at least the characteristics of color, texture, size 
and shape are quantifiable. As discussed above with respect to FIG. 2, each image 
in the database has a corresponding data representation in which the calculated 
values for these characteristics are stored for each defined region of interest 
(mask) in the image. For every mask, specific values for the image characteristics 
describing visual properties for the mask are calculated and entered into the data 
representation. Later, when a query is assembled, an object/thumbnail procedure 
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described below is employed to construct a description (a "sample image") of the 
images which a user wishes to retrieve from the image database, with, the query 
being constructed in terms of values of the image characteristics of interest. The 
query is used to find images in the database with image characteristic values that 
are similar to those included in the sample image. To do a query, the QBIC engine 
32 converts pictorial query information (e.g., the information from the thumbnails 
and their location) from the image query window into image characteristic values. 

The above cited passage of Barber teaches that there are several criteria, or image 
characteristics, which belong to various categories, such as color, shape, etc., from which one 
may choose in order to construct the image query. One chooses these image characteristics, 
represented as thumbnails, and these thumbnails are then dropped into an equation and this 
equation is used to analyze the graphical images. The Examiner is equating the categories 
themselves to critical factors and selecting a thumbnail from the category to prioritize the critical 
factor. 

However, this analogy is not correct. The process of creating a query by choosing 
thumbnails is not the same as "ranking the determined critical factors according to respective 
priorities set for each of the critical factors." Barber still only teaches choosing one priority 
(thumbnail) out of the set of priorities for the critical factor (category). As such, none of the 
other priorities are ever even considered when evaluating the image because Barber teaches that 
only the chosen thumbnail is put into the equation. Therefore, images are compared only against 
the chosen thumbnail and not evaluated against all the other priorities (thumbnails) for the given 
critical factor. Thus Barber, which teaches that sorting of images from best match to woist 
match is achieved by ranking the images' global scores, cannot teach the feature of ranking the 
determined critical factors according to respective priorities set for each of the critical factors, as 
recited in claim 1 of the present invention. 

The second cited passage, column 7, lines 2 through 26, also does not teach ranking the 
determined critical factors according to respective priorities set for each of the critical factors. 
Instead, the above cited passage of Barber merely teaches that a query is created by dragging and 
dropping thumbnails into a query window. The collection of deposited thumbnails creates an . 
object. The thumbnail descriptions are assembled into a query for the object, and the QBIC 
engine converts these descriptions into image characteristic values that are used for determining 
matches. 
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Additionally, Barber states in column 7, lines 3 through 43 that when the query is 
assembled, parameters are included 'In any order appropriate to the design": 

The QBIC engine 32 then assembles a query whose predicate includes a portion 
for each object contained in the query window. Each object portion of the query 
predicate includes, in any order appropriate to the design, parameters which 
correspond to the set of image characteristics. 

Therefore, as column 7, lines 3 through 43, teaches constructing the query in any order, it follows 
that Barber does not teach ranking the determined critical factors according to respective 
priorities set for each of the critical factors. 

Also, claim 1 recites the feature of generating a textual description of the set of graphical 
data, order ^ a^m-din ^ to the priorities of each of the respective critical factors . The Examiner 
points to column 14, lines 65 through 67, column 9, lines 57 through 61 , Figure 4, results list SO, 
and column 8, lines 46 through 55, as teaching generating the set of graphical data ordered 
according to the priorities of each of the respective critical factors. Column 14, lines 65 through 
67, is cited above. Column 9, lines 57 through 61, Figure 4 S results list 80, and column 8, lines 
46 through 55 are reproduced below: 

The order of the returned images is preferably sorted from best to worst match, 
and the number of images returned can be controlled by manipulation of the 
thumbnail attributes of weight and distance described above. 

(Barber, col. 9, lines 57-61) 

As the QBIC engine 32 executes the query, it assembles a results list 80 
which identifies images and ranks them according to their composite distance 
values. The results list 80 may include all images or only a prespecified number of 
the closest images. The results list 80 indexes to the images stored at 36, and is 
passed by the QBIC engine 32 to the window control 22. The window control 22 
retrieves the images in the results list from the image storage 36 and formats them 
appropriately for display in a results *window on the display 13. 

(Barber, col. 8, lines 46-55) 
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The above cited passages and Figure 4 of Barber do not teach generating the set of 
graphical data ordered according to the priorities of each of the respective critical factors, as 
alleged by the Examiner. Instead, column 14, lines 65 though 67, teaches that images are ranked 
by their global score and the images with the best scores are returned as a result of a query as 
previously discussed* Column 9, lines 57 through 61, merely teaches sorting the matches from 
best match to worst match- Column 8, lines 46 through 55, teaches that results list 80, of Figure 
4, contains a list of images for a query, sorted according to their composite distance value. A 
global score is the sum of the composite distance values for the various sectors of the image 
being checked (see Barber, col. 14, lines 47-64, cited above). The composite distance value is 
the sum of the Euclidean distances between the calculated image characteristics and the value for 
the corresponding thumbnail characteristics. The thumbnail characteristics, and consequently the 
categories from which the thumbnails are chosen, are not ranked or sorted in any order; they are 
simply built into the esquation depending on what thumbnails were selected by the user. 
Consequently, Barber does not teach generating a textual description of the set of graphical data, 
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ordered according to the priorities of each of the respective critical factors, as recited in claim 1 
of the present invention. 

Furthermore, simply choosing a thumbnail to put in the equation does not rank the 
category the thumbnail came from in comparison to the other categories. However, while the 
Examiner does not point to it, it should be noted that Barber does teach that the thumbnails 
themselves can be weighted. This still is different than the claimed feature of "ranking the 
determined critical factors according to respective priorities set for each of the critical factors," as 
the weighting is only relative to those other thumbnails (priorities) contained in the equation. As 
such, this weighting is not equivalent to ranking the categories (critical factors) themselves that 
the thumbnails (priorities) came from, 

Additionally, McLaughlin does not cure the deficiencies of Barber. McLaughlin does not 
teach or suggest the features missing from Barber, including ranking the determined critical 
factors according to respective priorities set for each of the critical factors* and generating a 
textual description of the set of graphical data, ordered according to the priorities of each of the 
respective critical factors, nor does the Examiner point to any portion of McLaughlin that teaches 
these features. 

The Examiner points to the following passage of McLaughlin as teaching generating a 
textual description of the set of graphical data: 

The web/text translator converts text, graphic icons, menu options, and 
communication links within a web site into equivalent forms of text data. A 
remote deaf user may read and select different options that previously were 
available only in an automated telephone service. A remote deaf user may access 
web sites across a PSTN and use the services of an automated system that were 
previously unavailable. 

(McLaughlin, col. 4, lines 3-9). . 

While McLaughlin may teach generating a textual description of the set of graphical data, 
McLaughlin still does not teach generating a textual description of the set of graphical data, 
ordered according to the priorities of each of the respective critical factors as recited in claim 1 of 
the present invention. No critical factors are either present or used in McLaughlin. As there are 
no critical factors, it follows that the textual description of the set of graphical data cannot be 
ordered according to the priorities of each of the respective critical factors. Therefore, 
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McLaughlin does not teach generating a textual description of the set of graphical data, ordered 
according to the priorities of each of the respective critical factors. 

Thus, even if one were to combine the teachings of Barber and McLaughlin, the resulting 
combination would not teach generating a textual description of the set of graphical data, ordered 
according to the priorities of each of the respective critical factors. Instead, the resulting 
combination would teach generating a textual description of the set of images, ranked from 
highest global score to lowest global score. Therefore, the combination of Barber and 
McLaughlin still would not reach the presently claimed invention. 

Furthermore, McLaughlin does not teach the other features missing from Barber, such as 
ranking the determined critical factors according to respective priorities set for each of the critical 
factors. As cited above and in the Abstract, McLaughlin teaches *\veb/text translator converts 
text, graphic icons, menu options, and communication links within a web site into equivalent 
forms of text data*" McLaughlin does not teach analyzing data to determine critical factors, nor 
does McLaughlin teach ranking critical factors. 

Therefore, for all the reasons set forth above, the combination of Barber with the 
McLaughlin reference would not reach the presently claimed invention as recited in claim 1. 
Claims 1, 8 and 15 are patentable over the cited references because the combination of the Barber 
reference with McLaughlin does not reach the presently claimed invention. 

Claims 2, 3, 6, 22, 23, 28 and 3 1 are dependent claims depending on claim 1, Claims 9, 
10, 13, 24 and 32 are dependent claims depending on claim 8. Claims 16, 17, 20, 25, 26 and 33 
are dependent claims depending on claim 15, As Appellants have already demonstrated that 
independent claims 1, 8 and 15 are patentable over the Barber and McLaughlin references, 
Appellants submit that dependent claims 2, 3, 6, 9, 10, 13, 16, 17, 20, 22-26 and 31-33 are 
patentable over the Barber and McLaughlin references at least by virtue of depending from an 
allowable claim. Additionally, claims 2, 3, 6, 9, 10, 13, 16, 17, 20, 22-26 and 31-33 claim other 
additional combinations of features not suggested by the reference. 

Al. CLAIMS 2, 9, and 16 

For instance, claims 2, 9, and 16 recite wherein the set of critical factors and the textual 
description are selected according to a selected mode. The Examiner alleges that this feature is 
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found in the following cited sections of Barber: 

The example image window 90 includes a image query (iq) icon 89 on which user 
may "click 11 to begin an image query. Alternately, a query object maybe dragged 
to the image query icon 89 or an appropriate command may typed into a command 
line (not shown)* 

(Barber, col. 9, lines 1-4). 

In order to generate a query based on the thumbnails 100 and 106 dropped 
in the example image window 90, a RUN QUERY option is selected, 
(Barber, col. 9, lines 41-43), 

The first passage cited above, column 9, lines 1 through 4, teaches that in order to begin 
building an image query, a user has three options: click the image query icon, drag a query object 
to the image queryicon, or type a command into a command line. The second passage cited 
above, column 9, lines 41 through 43, teaches that in order to execute, or perform, the query that 
the user has just constructed, the user needs to select the RUN QUERY option. Neither of these 
passages teaches wherein the set of critical factors and the textual description axe selected 
according to a selected mode. At best, the above cited passages teach that there is a mode of 
operation in which the user can choose, manually, what the critical factors are. Nowhere do the 
cited passages teach that the set of critical factors and the textual description are selected 
according to a selected mode. Thus, Barber does not teach the feature of wherein the set of 
critical factors and the textual description are selected according to a selected mode, as recited in 
claims 2, 9, and 16 of the present invention. 

A2. CLAIMS 6, 13, and 20 

Claims 6, 13, and 20 recite wherein said priorities of each of the respective critical factors 
is determined in accor ^nm wit h said selected mode . The Examiner alleges that this feature is 
found in column 9, lines 1 through 4, of Barber, cited above. As was previously discussed, 
column 9, lines 1 through 4, teaches that in order to begin building an image query, a user has 
three options: click the image query icon, drag a query object to the image query icon, or type a 
command into a command line. The passage does not teach anything about the priorities of 
critical factors or that said priorities of each of the respective critical factors is determined in 
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accordance with said selected mode. Thus, Barber does not teach the feature of wherein said 
priorities of each of the respective critical factors is determined in accordance with said selected 
mode, as recited in claims 6, 13 and 20 of the present invention. 
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B. GROUND OF REJECTION 2 (Claims 3, 10 and 17) 

The Office Action rejects claims 3, 10, and 17 under 35 ILS.C. § 103(a) as being 
unpatentable over Barber in view of McLaughlin et al and further in view of applicant's admitted 
prior art. This rejection is respectfully traversed. 

As to claims 3, 10, and 17 the Office Action states: 

As per claims 3, 10 and 17, modified Barber does not disclose the mode is 
selected according to a URL associated with the set of graphical data. This 
feature is taught by applicant's admitted prior art. It would have been obvious to 
an artisan at the time of the invention to select the mode according to a URL 
associated with the set of graphical data in modified Barber's system since it 
would make the system adaptable to the Wcb/lnternet technology. 

Final Office Action dated September 22, 2005, page 4. 

Claims 3, 10, and 17 are dependent claims depending from independent claims 1, 8, and 
1 5, respectively. The Barber reference still does not teach or suggest all the claim limitations in 
claims 3, 10, and 17, as argued in the response to the rejection of claims 1,8, and 1 5 above. 

Additionally, neither McLaughlin nor Appellants 7 admitted prior art cures the 
. deficiencies of Barber. Neither McLaughlin nor Appellants' admitted prior art teaches the 
features missing from Barber, including analyzing a set of graphical data to determine a set of 
critical factors present in the graphical data to form determined critical factors, ranking the 
determined critical factors according to respective priorities set for each of the critical factors, 
and generating a textual description of the set of graphical data, ordered according to the 
priorities of each of the respective critical factors, nor does the Examiner point to any portion of 
McLaughlin or Appellants' admitted prior art that teaches these features. 

Thus claims 3,10, and 17 are patentable over the cited references because the 
combination of Barber in view of McLaughlin and further in view of applicant' s admitted prior 
art does not reach the presently claimed invention. The features relied upon as taught in the 
Barber reference are not taught or suggested by that reference* bb explained above. As a result, a 
combination of these references does not teach the claimed invention in claims 3, 10, and 17. 

In view of the above, Appellants submit that dependent claims 3, 10, and 17 are not 
taught or suggested by Barber in view of McLaughlin and further in view of applicant's admitted 
prior art. Appellants have already demonstrated independent claims 1, 8 and 15 to be in 
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condition for allowance, Appellants respectfully submit that claims 3» 1 0 a and 17 are also 
allowable at least by virtue of depending from an allowable claim, 

C. GROUND OF RE JECTION 3 (Claims 4, 5, 11, 12, 18 and 19) 

The Office Action rejects claims 4, 5, 11, 12, 18, and 19 under 35 U.S.C. § 103(a) as 
being unpatentable over Barber in view of McLaughlin and further in view of Hasser et al. (U.S. 
Patent No. 5,736,978) (hereinafter cc Hasser"). This rejection is respectfully traversed. 

As to claims 4, 5, 11, 12, 18 and 19 the Office Action states: 

As per claims 4 and 5, modified Barber does not disclose the step of 
generating said textual description further comprises the step of generating said 
contextual rendition in an aural format and in a tactile format. Hasser teaches the 
communication of graphic data provided by tactile sensing and audio related user 
aids (col, 4, liens 55-63). It would have been obvious to an artisan at the time of 
the invention to use the teaching from Hasser of providing the communication of 
graphic data by tactile sensing and audio related user aids in Barber's system since 
it would enable modified Barber's system to be used by sight impaired people. 

Final Office Action dated September 22, 2005 s page 5. 

Claims 4, 5, 1 1, 12> 18, and 19 are dependent claims depending from independent claims 
l p 8, and 15, respectively. Although Hasser may teach communication of graphic data provided 
by tactile sensing and audio related user aids, the features relied upon as taught in the Barber 
reference are not taught or suggested by that reference, as explained above in the response to the 
rejection of claims 1, 8, and 15 above. 

Furthermore, neither McLaughlin nor Hasser cures the deficiencies of Barber. Neither 
McLaughlin nor Hasser teaches the features missing from Barber, including analyzing a set of 
graphical data to determine a set of critical factors present in the graphical data to form 
determined critical factors, ranking the determined critical factors according to respective 
priorities set for each of the critical factors, and generating a textual description of the set of 
graphical data, ordered according to the priorities of each of the respective critical factors, nor 
does the Examiner point to any portion of McLaughlin or Hasser that teaches these features. 

In view of the above, Appellants submit that dependent claims 4, 5. 11, 12, 18, and 19 are 
not taught or suggested by Barber in view of McLaughlin and further in view of Hasser. 
Appellants have already demonstrated independent claims 1, 8 and 15 to be in condition for 
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allowance. Appellants respectfully submit that claims 4 f 5, 1 1, 12, 18, and 19 are also allowable 
at least by virtue of depending from an allowable claim. 

D. GROUND OF REJECTION 4 (Claims 7, 14 and 21) 

The Office Action rejects claims 7, 14 and 21 under 35 U.S.C. § 103(a) as being 
unpatentable over Barber in view of McLaughlin and further in view of Discolo et aL (U.S. 
Patent No. 6,370,566) (hereinafter ''DiscolD 1 *). This rejection is respectfully traver&ed. 

As to claims 7, 14, and 21 the Office Action states: 

As per claims 7, 14 and 21 s which is dependent on claim 1, modified 
Barber does not disclose the step of generating said textual description of the set 
of graphical data includes generating said textual description in accordance with 
one or more textual templates. Discolo discloses that at col. 22, lines 31-32. It 
would have been obvious to an artisan at the time of the invention to use the 
teaching from Discolo of generating the textual description in accordance with 
one or more textual templates in modified Barber's system since it would make 
the process run fester by retrieving the textual description directly from the textual 
templates, 

Final Office Action dated September 26, 2005, pages 5-6. 

Claims 7, 14, and 21 are dependent claims depending from independent claims 1, 8, and 
15, respectively. The Barber reference still does not teach or suggest all the claim limitations in 
claims 7, 14, and 21, as argued in the response to the rejection of claims 1, 8, and 1 5 above. 

Furthermore, neither McLaughlin nor Discolo cures the deficiencies of Barber, Neither 
McLaughlin nor Discolo teaches the features missing from Barber, including analyzing a set of 
graphical data to determine a set of critical factors present in the graphical data to form 
determined critical factors, ranking the determined critical factors according to respective 
priorities set for each of the critical factors, and generating a textual description of the set of 
graphical data, ordered according to the priorities of each of the respective critical factors, nor 
does the Examiner point to any portion of McLaughlin or Discolo that teach these features. 

Thus, claims 7, 14, and 21 are patentable over the cited references because the 
combination of Barber in view of McLaughlin and further in view of Discolo does not teach the 
presently claimed invention. Appellants have already demonstrated independent claims 1, 8, and 
1 5 to be in condition for allowance. Appellants respectfully submit that claims 7, 14, and 21 are 
also allowable at least by virtue of depending from an allowable claim. 
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E. GROUND OF REJECTION 5 (Claims 27-30) 

The Office Action rejects claims 27-30 under 35 U.S.C. § 103(a) as being unpatentable 
over Barber in view of McLaughlin and further in view of applicant's admitted prior art. This 
rejection is respectfully traversed. 

As to claims 27-30 the Office Action states: 

As per claim 27, modified Barber does not disclose the graphical data is 
selected from a group of GIF, JPEG, and PNG type data format. Applicant's 
admitted prior art disclose that in page 3, lines 4-6- It would have been obvious to 
an artisan at the time of the invention to apply the teaching from, the applicant's 
admitted prior art of selecting graphical data from a group of GIF, JPBG, and 
PNG type data format in modified Barber's system since it would enable the 
system to work with different types of data formats, 

As per claim 28, modified Barber does not disclose the set of critical 
factors includes characteristics of data illustrated in a displayed multi-dimensional 
graph. Applicant's admitted prior art disclose that in page 3, lines 10-16. It 
would have been obvious to an artisan at the time of the invention to apply the 
teaching from the applicant's admitted prior art of illustrating the set of critical 
factors in a displayed multi-dimensional graph in modified Barber' s system since 
it would help a user easily visualize critical factors on the multi-dimensional 
graph. 

Final Office Action dated September 22, 2005, page 6. 

Claims 27-30 are dependent claims depending from independent claims 1, 8, and 15, 
respectively. The Barber reference still does not teach or suggest all the claim limitations in 
claims 27-30, as argued in the response to the rejection of claims 1, 8, and 15 above- 
Furthermore, neither McLaughlin nor Appellants' admitted prior art cures the deficiencies 
of Barber. Neither McLaughlin nor Appellants' admitted prior art teaches the features missing 
from Barber, including analyzing a set of graphical data to determine a set of critical factors 
present in the graphical data to form determined critical factors, ranking the determined critical 
factors according to respective priorities set for each of the critical factors, and generating a 
textual description of the set of graphical data, ordered according to the priorities of each of the 
respective critical factors, nor does the Examiner point to any portion of McLaughlin or 
Appellants' admitted prior art that teaches these features* 
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In view of the above, Appellants submit that dependent claims 27-30 are not taught or 
suggested by Barber in view of McLaughlin and further in view of applicant's admitted prior art. 
Appellants have already demonstrated independent claims 1, 8, and 15 to be in condition for 
allowance. Appellants respectfully submit that claims 27-30 are also allowable at least by virtue 
of depending from an allowable claim. 
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CLAIMS APPENDIX 
The text of the claims involved in the appeal reads; 

1 . A method in a data processing system for presenting graphical data to a user, comprising 
the steps of; 

analyzing a set of graphical data to determine a set of critical factors present in the 
graphical data to form determined critical factors; 

ranking the determined critical factors according to respective priorities set for each of the 
critical factors; and 

generating a textual description of the set of graphical data, ordered according to the 
priorities of each of the respective critical factors. 

2. The method as recited in claim 1, wherein the set of critical factors and the textual 
description are selected according to a selected mode. 

3. The method as recited in claim 2, wherein the mode is selected according to a URL 
associated with the set of graphical data. 

4. The method of claim 1 wherein said step of generating said textual description further 
comprises the step of generating said textual rendition in an aural format. 

5. The method of claim 1 wherein said step of generating said textual description further 
comprises the step of generating said textual rendition in a tactile format. 
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6. The method of claim 2 wherein said priorities of each of the respective critical factors is 
determined in accordance with said selected mode. 

7. The method of claim 1 whexein said step of generating said textual description of the set 
of graphical data includes generating said textual description in accordance with one or more 
textual templates. 

8. A computer program product embodied in a tangible storage medium, the program 
product for presenting graphical data, the program product including a program of instructions 
for performing the steps of: 

analyzing a set of graphical data to determine a set of critical factors present in the 
graphical data to form determined critical factors; 

ranking the determined critical factors according to respective priorities set for each of the 
critical factors; and 

generating a textual description of the set of graphical data, ordered according to the 
priorities of each of the respective critical factors. 

9. The program product of claim 8 wherein the set of critical factors and the textual 
description are selected according to a selected mode.[[.]] 

10. The program product of claim 9 wherein the mode is selected according to a URL 
associated with the set of graphical data. 
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1 1 . The program product of claim 8 wherein said program of instruction for performing the 
step of generating said textual description further comprises a program of instruction for 
performing the step of generating said textual description in an aural format. 

12. The program product of claim 8 said program of instruction for performing the step of 
generating said textual description comprises a program of instruction for performing the step of 
generating said textual description in a tactile format. 

1 3 . The program product of claim 9 wherein said priorities of each of the respective critical 
factors is determined in accordance with said selected mode. 

14. The program product of claim 8 wherein said step of generating said textual description 
of the set of graphical data includes generating said textual description in accordance with one or 
more textual templates. 

1 5. A data processing system comprising: 

circuitry operable for analyzing a set of graphical data to determine a set of critical factors 
present in the graphical data to form determined critical factors; 

circuitry operable for ranking the determined critical factors according to respective 
priorities set for each of the critical factors; and 

circuitry operable for generating a textual description of the set of graphical data* ordered 
according to the priorities of each of the respective critical factors. 

16. The system as recited in claim 15, wherein the set of critical factors and the textual 
description are selected according to a selected mode. 

(Appeal Brief Page 28 Of 33) 
ianaJfirraiari et al. - 09/842, 835 

PAGE 30135 ' RCVD AT 2/2112006 11:38:14 AM [Eastern Standard Time] » SVR:USPTO-EFXRF-6/35 * DNIS:2738300 ' CSID:972 385 7766 ' DURATION (mm-ss):0844 



Feb. 21 2006 1 1 : 50RH YEE 8, ASSOCIATES, P,C. 



C972J 385-77GG 



31 



BEST AVAILABLE COPY 

17. The system as recited in claim 16, wherein the mode is selected according to a URL 
associated with the set of graphical data. 

1 8. The system of claim 1 5 wherein said circuitry operable for generating said textual 
description further comprises circuitry operable for generating said textual rendition in an aural 
format 

19. The system d'f claim 15 wherein said circuitry operable for generating said textual 
description further comprises circuitry operable for generating said textual rendition in a tactile 
format. 

20. The system of claim 16 wherein said priorities of each of the respective critical factors is 
determined in accordance with said selected mode, 

21 . The system of claim 15 wherein said circuitry operable for generating said textual 
description of the set of graphical data includes circuitry operable for generating said textual 
description in accordance with one or more textual templates. 

22. The method as recited in claim 1, wherein the graphical data further comprises data in a 
format that produces a non-textual image on a display screen. 

23* The method as recited in claim 22, wherein the generating step uses image analysis 
software for converting the graphical data into a textual description of the graphical data. 
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24. The program product of claim 8, wherein the graphical data further comprises data in a 
format that produces a non-textual image on a display screen* 

25 . The system as recited in claim 1 5 , wherein the graphical data further comprises data in a 
format that produces a non-textual image on a display screen. 

26. The system as recited in claim 15, wherein the graphical data further comprises data in a 
format that produces a predominately non-textual image on a display screen. 

27. The program product as recited in claim 8, wherein the graphical data is selected from a 
group of GIF, JPEG, and PNG type data formats. 

28. The method as recited in claim 1, wherein the set of critical factors includes 
characteristics of data illustrated in a displayed multi-dimensional graph. 

29. The computer program product as recited in claim 8, wherein the set of critical factors 
includes characteristics of data illustrated in a displayed multi-dimensional graph* 

30. The system as recited in claim 1 5, wherein the set of critical factors includes 
characteristics of data illustrated in a displayed multi-dimensional graph. 

31. The method as recited in claim 1 , wherein the textual description of the set of graphical 
data describes in words an illustrated description of the graphical data. 

32. The computer program product as recited in claim 8 S wherein the textual description of 
the set of graphical data describes in words an illustrated description of the graphical data, 
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33. The system as recited in claim 15, wherein the textual description of the set of graphical 
data describes in words an illustrated description of tie graphical data. 
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EVIDENCE APPENDIX 
There is no evidence to be presented. 
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RELATED PROCEEDINGS APPENDIX 

There are no related proceedings. 
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